Text copied to clipboard!
Title
Text copied to clipboard!Data Engineer
Description
Text copied to clipboard!
We are looking for a Data Engineer to join our growing team of analytics experts. The ideal candidate will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams. The Data Engineer will support our software developers, database architects, data analysts, and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
The right candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products. The ideal candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.
Key responsibilities include creating and maintaining optimal data pipeline architecture, assembling large, complex data sets that meet functional and non-functional business requirements, identifying, designing, and implementing internal process improvements, and building the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
The Data Engineer will also be responsible for working with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs. They will create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
This role requires strong experience with big data tools, relational SQL and NoSQL databases, data pipeline and workflow management tools, and cloud services. A successful candidate will have a track record of manipulating, processing, and extracting value from large disconnected datasets and a strong project management and organizational skills.
Responsibilities
Text copied to clipboard!- Design, construct, install, and maintain large-scale processing systems
- Build high-performance algorithms, predictive models, and prototypes
- Ensure systems meet business requirements and industry practices
- Integrate new data management technologies and software engineering tools
- Collaborate with data architects, modelers, and IT team members
- Develop set processes for data mining, data modeling, and data production
- Create custom software components and analytics applications
- Monitor performance and advise on infrastructure changes
- Prepare data for prescriptive and predictive modeling
- Use large data sets to resolve business issues
Requirements
Text copied to clipboard!- Bachelor’s degree in Computer Science, Engineering, or related field
- 3+ years of experience in a Data Engineer role
- Advanced working SQL knowledge and experience with relational databases
- Experience building and optimizing big data pipelines and architectures
- Strong analytic skills related to working with unstructured datasets
- Experience with big data tools like Hadoop, Spark, Kafka, etc.
- Proficiency in Python, Java, or Scala
- Experience with cloud services (AWS, Azure, GCP)
- Knowledge of data pipeline and workflow management tools
- Excellent problem-solving and communication skills
Potential interview questions
Text copied to clipboard!- Can you describe your experience with building data pipelines?
- What big data tools have you worked with?
- How do you ensure data quality and integrity?
- Describe a challenging data engineering project you worked on.
- What cloud platforms are you familiar with?
- How do you handle unstructured or messy data?
- What programming languages are you most comfortable with?
- How do you collaborate with data scientists and analysts?
- What’s your approach to optimizing data workflows?
- Have you worked in Agile or DevOps environments?
